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Abstract —  This  is  the  second  paper  in  a  series  of  two  that  describe  our  research  in  intelligent  energy  management  in  a  hybrid  electric 
vehicle  (HEV).  Energy  management  in  Hybrid  Electric  Vehicles  (HEV)  has  been  actively  studied  recently  because  of  its  potential  to 
significantly  improve  fuel  economy  and  emission  control.  Because  of  the  dual-power-source  nature  and  the  complex  configuration  and 
operation  modes  in  a  HEV,  energy  management  is  more  complicated  and  important  than  in  a  conventional  vehicle.  Most  of  the  existing 
vehicle  energy  optimization  approaches  do  not  incorporate  knowledge  about  driving  patterns  into  their  vehicle  energy  management 
strategies.  Our  approach  is  to  use  machine  learning  technology  combined  with  roadway  type  and  traffic  congestion  level  specific 
optimization  to  achieve  quasi-optimal  energy  management  in  hybrid  vehicles. 

In  this  series  of  two  papers,  we  present  a  machine  learning  framework  that  combines  Dynamic  Programming  with  neural  networks  to 
learn  about  roadway  type  and  traffic  congestion  level  specific  energy  optimization,  and  an  integrated  online  intelligent  energy  controller 
to  achieve  quasi-optimal  power  management  in  hybrid  vehicles.  In  the  first  paper  we  presented  a  machine  learning  framework, 
ML_EMO_HEV,  developed  for  learning  the  knowledge  about  energy  optimization  in  an  HEV.  The  framework  consists  of  machine 
learning  algorithms  for  predicting  driving  environments  and  for  generating  optimal  power  split  of  the  HEV  system  for  a  given  driving 
environment.  Experiments  are  conducted  to  evaluate  these  algorithms  using  a  simulated  Ford  Escape  Hybrid  vehicle  model  provided  in 
PSAT  (Powertrain  Systems  Analysis  Toolkit). 

In  this  second  paper,  we  present  three  online  intelligent  energy  controllers,  IEC_HEV_SISE,  IEC_HEV_MISE,  and 
IEC_EIEV_MIME.  All  three  online  intelligent  energy  controllers  were  trained  within  the  machine  learning  framework,  ML_EMO_HEV 
were  trained  to  generate  the  best  combination  of  engine  power  and  battery  power  in  real-time  such  that  the  total  fuel  consumption  over 
whole  driving  cycle  is  minimized  while  still  meeting  the  driver’s  demand  and  the  system  constraints  including  engine,  motor,  battery, 
and  generator  operation  limits.  The  three  online  controllers  were  integrated  into  the  Ford  Escape  Hybrid  vehicle  model  for  online 
performance  evaluation.  Based  on  their  performances  on  10  test  drive  cycles  provided  by  the  PSAT  library,  we  can  conclude  that  the 
roadway  type  and  traffic  congestion  level  specific  machine  learning  of  optimal  energy  management  is  effective  for  in-vehicle  energy 
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control.  The  best  controller,  IEC_HEV_MISE,  trained  with  the  optimal  power  split  generated  by  the  DP  optimization  algorithm  with 
multiple  initial  SOC  points  and  single  ending  point  can  provide  fuel  saving  range  from  5%  through  19%. 

In  conclusion,  together  these  two  papers  cover  the  innovative  technologies  for  modeling  power  flow,  mathematical  background  of 
optimization  in  energy  management,  and  machine  learning  algorithms  for  generating  intelligent  energy  controllers  for  quasi-optimal 
energy  flow  in  a  power  split  HEV. 

Index  Terms —  Fuel  economy,  machine  learning,  energy  optimization,  HEV  power  management 
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I.  Introduction 

Power  management  in  Hybrid  Electric  Vehicles  (HEV)  has  been  actively  studied  recently  because  of  its  potential  to  significantly 
improve  fuel  economy  and  emission  control  [1-10].  Because  of  the  dual -power-source  nature  and  the  complex  configuration  and 
operation  modes  in  an  HEV,  power  management  is  more  complicated  but  also  more  important  than  in  a  conventional  vehicle.  Most 
of  the  existing  vehicle  power  optimization  approaches  do  not  incorporate  knowledge  about  driving  patterns  into  their  vehicle  power 
management  strategies.  Our  approach  is  to  use  machine  learning  technology  combined  with  energy  optimization  specific  to  Sierra 
Facility  Specific  drive  cycles  [11-15]  to  achieve  quasi-optimal  power  management  in  hybrid  vehicles.  The  11  standard  Sierra 
Facility  Specific  drive  cycles  are  referred  as  roadway  types  and  traffic  congestion  levels.  In  a  series  of  two  papers,  we  present  a 
machine  learning  framework  that  combines  Dynamic  Programming  [8-10]  with  machine  learning  to  learn  about  roadway  type  and 
traffic  congestion  level  specific  energy  optimization,  and  an  integrated  online  intelligent  energy  controller  to  achieve  quasi-optimal 
power  management  in  hybrid  vehicles.  In  the  first  paper,  we  presented  the  machine  learning  architecture  of  the  intelligent  vehicle 
power  management  system,  ML_EMO_HEV.  This  framework  consists  of  two  parts.  First  it  contains  neural  learning  algorithms 
for  predicting  roadway  types  and  traffic  congestion  levels  and  driving  trends  based  on  the  vehicle's  recent  driving  history.  Second, 
it  contains  roadway  type  and  traffic  congestion  level  specific  machine  learning  algorithms  for  the  purpose  of  optimizing  the  power 
split  between  the  propulsion  system  components  within  the  HEV.  This  second  set  of  machine  learning  algorithms  are  trained  using 
optimal  solutions  from  offline  Dynamic  Programming  analysis  of  roadway  type  and  traffic  congestion  level. 

In  this  second  paper,  we  present  an  online  intelligent  energy  controller  (IEC)  for  HEV,  IEC_HEV.  IEC_HEV  is  developed 
under  the  ML_EMO_HEV  framework,  and  trained  by  the  machine  learning  algorithms  presented  in  the  first  paper  of  this  series. 
We  will  also  cover  the  critical  issues  related  to  the  integration  of  IEC_HEV  to  a  vehicle  control  system,  and  system  performance 
analysis  on  various  drive  cycles  using  different  settings  of  learning  parameters.  The  experiments  were  conducted  with  four 
different  IEC_HEV  models  that  were  implemented  inside  PSAT. 

The  organization  of  the  remainder  of  the  paper  is  as  follows.  Section  II  introduces  energy  dynamics  in  a  power-split  HEV  model. 
Section  III  presents  an  intelligent  online  energy  controller,  IEC_HEV,  and  the  online  implementation  of  IEC_HEV  in  a  power-split 
HEV  model.  Section  IV  performs  experiments  to  evaluate  different  versions  of  IEC_HEV  with  various  cycles  provided  in  PSAT. 


Section  V  concludes  this  paper. 
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II.  ENERGY  DYNAMICS  IN  A  POWER  SPLIT  HEV  MODEL 


A  power  split  HEV  has  two  power  sources  that  are  connected  to  the  driveline  using  a  planetary  gear  set  as  shown  in  Figure  1 . 
The  power  train  consists  of  the  engine,  generator,  motor,  battery  and  planetary  gear  set.  The  planetary  gear  set  is  the  key  device  in 
the  power  split  HEV  power  train  that  connects  the  engine,  generator,  and  motor  as  a  power  split  device.  The  engine  is  connected  to 
the  carrier  gear,  the  generator  is  connected  to  the  sun  gear  and  the  motor  is  linked  to  the  ring  gear  of  the  planetary  gear  set.  The 
planetary  gear  set  transmits  power  between  the  engine,  motor,  generator  and  front  wheels.  The  vehicle  can  be  propelled  by  either 
one  or  both  of  two  power  sources.  The  first  power  source  is  the  combined  engine  and  generator.  This  combination  can  be  used  to 
provide  mechanical  power  to  the  driveline  directly  or  to  provide  electrical  power  to  the  high  voltage  system.  The  second 
propulsion  system  consists  of  the  motor,  possibly  in  combination  with  generator.  The  motor  (and  potentially  generator)  can  be 
used  to  propel  the  vehicle  by  drawing  electrical  energy  from  the  battery  or  by  using  the  high  voltage  energy  produced  by  the 
combined  engine  and  generator. 

Because  of  the  kinematic  properties  of  the  planetary  gear  set,  the  engine  speed  can  be  decoupled  from  the  vehicle  speed,  which 
provides  a  great  potential  for  optimizing  engine  efficiency  [16-17].  The  decoupled  engine  speed  and  the  amount  of  battery  power 
that  is  charged  or  discharged  are  the  two  degrees  of  freedom  used  by  IECJHEV  for  vehicle  energy  optimization.  IEC_HEV,  the 


Trans-Axle 


Driver  Inputs  -  Acceleration  Pedal  Position, 

Brake  Pedal  Position,  and  Gear  Position 
O.W.C  -  On <5  Way  Hutch 


VSC  —  Vehicle  System  Controller 
TCM  —  Transmission  Control  Module 
BCM—  Battery  Control  Module 
BS\I-  Brake  System  Module 


Figure  1.  Power  Split  HEV  configuration  into  VSC 


5 


online  energy  control  is  a  component  in  the  Vehicle  System  Controller  (VSC).  The  VSC  for  this  configuration  must  manage  the 
powertrain  control  in  order  to  maintain  a  proper  level  of  charge  in  the  battery.  However,  since  two  power  sources  are  available  to 
propel  the  vehicle,  the  VSC  in  this  configuration  has  the  additional  responsibility  of  coordinating  the  two  power  sources  to  properly 
deliver  power  to  the  wheel  [18].  Based  on  the  optimal  engine  speed  and  battery  power  generated  by  the  IEC_HEV,  the  VSC  derives 
the  speed  and  power  of  the  other  components  (engine,  motor,  and  generator)  based  on  the  following  kinematics  and  dynamics  of 
the  power-split  HEV  system  [17-19], 

The  kinematics  of  a  planetary  gear  set  defines  the  speed  relationship  between  the  carrier,  cocarrier,  sun  gear,  coslm  and  ring  gear, 
C0ring  as  shown  in  the  equation  below  [19]: 


OX, 
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1  ring  '  s 


-  Wsun  + 
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where  Nsun  is  the  number  of  teeth  of  sun  gear  and  Nri„g  is  the  number  of  teeth  of  ring  gear.  The  fixed  torque  split  between  the  carrier, 
t carrien  sun  gear,  xslm  and  ring  gear,  rrmg  is  defined  as  below: 
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Because  the  planetary  gear  set  acts  as  a  power  split  device,  the  total  of  the  powers  of  the  elements  in  the  device  sum  to  zero.  The 
following  equations  are  derived  based  on  the  power  flow  illustrated  in  Figure  2. 


V. 


Figure  2.  Power  flow  in  a  Split  HEV  configuration  with  arrows  indicating  positive  power. 
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The  relationship  between  the  power  delivered  at  the  carrier,  P carrier,  the  power  at  the  ring  gear,  Prjng,  and  the  power  at  the  sun  gear, 
Psu„  is  described  in  the  equation  below: 


p  +p  +p  =0  . 

carrier  sun  ring 


(3) 


In  addition,  the  power  across  the  electrical  power  net  can  be  expressed  as  follows: 

P  =-(P  +P  +P  )=-(P  +P  +P  +P  +  P  )  (4) 

batt  n  gen_e  mot_e  L  '  n  gen  gen_loss  mot  mot_loss  L  '  '  ' 

where  Pbatt  is  power  output  at  the  battery  terminal,  Pgen_e  is  the  electrical  power  at  the  generator,  Pgenjoss  is  the  generator  power  loss 
during  power  conversion  using  the  generator  efficiency,  r/g  ,  Pmot  e  is  the  electrical  power  at  the  motor,  Pmotjoss is  the  motor  power 


loss  during  power  conversion  using  the  motor  efficiency,  Ij m  ,  and  PL  represents  the  power  of  the  electrical  load.  Based  on  equation 

(4)  and  the  fact  that  the  power  at  the  final  drive  is  provided  by  the  ring  gear  power  and  motor  power,  equation  (5)  shows  how  the 
engine  power,  Peng  is  represented  using  the  power  split  dynamics: 


P  -  p  +  f- P  f=P  -  P  +P  =  P  +  P  +  P  +  P  +  P 

eng  ring  V  gen-'  ring  gen_e  gen_loss  ring  batt  L  mot_e  gen_loss 

—  P  +P  +P+P  +P  +P  —  P  +P  +P+P  +P 

ring  batt  L  mot  mot  _loss  gen_loss  drive  _sh  batt  L  mot_loss  gen_loss 


(5) 


The  ring  gear  speed,  conng,  and  the  motor  speed,  wmotor,  are  coupled  to  the  driveshaft  speed,  codrive-sh,  and  correspondingly,  the 
vehicle  speed,  V5 .  Based  on  this  constraint,  from  the  quasi-optimal  engine  speed  coeng  =  (cocarrjel)  generated  by  the  IEC-HEV,  the 

VSC  derives  the  sun  gear  speed,  cosun.  The  quasi-optimal  battery  power,  Pba„,  generated  by  IEC_HEV  is  used  by  VSC  to  determine 
the  quasi-optimal  motor  torque  and  power.  Thus  a  quasi-optimal  power  split  is  achieved. 

III.  INTELLIGENT  ONLINE  ENERGY  CONTROL 

In  general  HEV  operations,  the  engine  can  be  operated  in  three  states  as  illustrated  in  Figure  3:  “engine  off’,  “engine  start-up”, 
and  “engine  on.”  When  the  engine  is  in  the  “off’  state,  it  goes  to  the  “start-up”  state  when  the  driver’s  power  demand  is  bigger  than 
a  threshold.  The  threshold  value  is  typically  calculated  based  on  the  current  battery  state  of  charge,  SOC.  Figure  4  shows  such  a 
threshold  function  for  the  Ford  Escape  provided  in  the  PSAT  modeling  environment.  Once  the  engine  speed  is  higher  than  the 
engine  idle  speed  (e.g.  83.76  rad/sec)  then  the  engine  goes  to  the  ‘engine-on’  state,  and  stays  there  until  the  driver’s  power  demand 
becomes  negative.  Then  the  engine  is  turned  off,  i.e.  it  enters  the  “engine  off’  state  from  "engine  on"  state.  The  Intelligent  Energy 
Controller  for  HEV,  IEC_HEV,  is  only  operational  during  the  "engine  on"  state.  The  conditions  used  in  Figure  3  to  illustrate  the 
state  transitions  were  derived  from  the  Ford  Escape  HEV  model  provided  by  PSAT  library. 
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Figure  3.  State  diagram  of  engine  On/Off  in  the  HEV. 
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Figure  4.  Threshold  of  driver  power  demand  as  a  function  of  SOC  for  engine  turning  on. 


When  the  engine  is  in  the  “on”  state,  the  power  split  between  the  mechanical  and  the  electrical  path  at  every  time  instance 
determines  the  fuel  economy  over  the  entire  drive  cycle.  IEC-HEV  is  designed  and  trained  to  generate,  at  every  time  instance,  an 
intelligent  power  split  between  the  two  energy  sources,  mechanical  power  from  the  engine  and  the  electrical  power  from  the 
battery,  so  the  fuel  consumption  over  the  entire  drive  cycle  is  optimized  and,  at  the  same  time,  the  driver’s  demand  and  the  system 
constraints  (engine,  motor,  battery,  and  generator  limits)  are  fully  met.  Because  global  optimization  algorithms  such  as  Dynamic 
Programming  require  the  knowledge  about  the  entire  drive  cycle  being  known  a  priori,  therefore  these  methods  are  not  directly 
applicable  to  on-line  implementation  since  the  future  driving  information  is  generally  unknown  during  vehicle  operations  [19-21], 
Our  approach  is  to  use  machine  learning  technology  to  predict  the  short  term  driving  environment,  i.e.  roadway  type  and  traffic 
congestion  level  and  driving  trend,  and  apply  the  appropriate  energy  split  in  the  real  time  operation  of  a  drive  cycle.  We  presented, 
in  Part  I  of  this  paper  series,  the  machine  learning  algorithms  for  training  neural  networks  to  emulate  the  optimal  power  control  of 
an  HEV  generated  by  DP,  and  the  neural  networks  to  predict  driving  environment,  which  includes  roadway  type  and  traffic 
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congestion  level  the  vehicle  is  currently  on  and  the  driving  trend  of  the  vehicle.  IEC-HEV  is  an  integrated  system  of  these  neural 
networks.  Figure  5  illustrates  the  major  components  in  the  intelligent  energy  controller  IEC-HEV.  IEC_HEV  consist  of  three 
types  of  neural  networks:  NN_RT&TC,  a  neural  network  trained  to  predict  roadway  type  and  traffic  congestion  level  based  on  the 
vehicle  dynamics,  NN_DT,  a  neural  network  trained  to  predict  driving  trend,  and  the  2 K  power  split  neural  networks  for  K  standard 
roadway  types  and  traffic  congestion  levels.  As  discussed  in  the  first  paper  of  the  series,  we  use  in  this  research  the  1 1  Sierra  FS 
drive  cycles  as  the  standard  description  of  roadway  types  and  traffic  congestion  levels.  Therefore  we  have  K  =  11.  For  the 
convenience  of  description  we  denote  the  11  standard  roadway  types  and  traffic  congestion  levels  as  Rh  1=1,. ..,11.  For  each 
roadway  type  and  traffic  congestion  level  Rn  two  neural  networks  have  been  developed  and  trained,  NN '  to  predict  the  optimal 

**bat 

battery  power  Pbatt,  and  AW'  to  predict  the  optimal  engine  speed  co The  training  details  of  all  these  neural  networks  are 

presented  in  the  Part  I  of  this  paper  series.  The  output  from  the  power  split  neural  networks  are  used  to  generate  the  optimal  engine 
power,  Peng  (t)  following  the  equation  (1)~(5).  Both  the  engine  power  Peng  ( t )  and  the  engine  speed  coeng(t )  are  being  checked  to 
make  sure  they  are  within  the  constraints  such  as  being  less  than  the  maximum  engine  power  and  the  maximum  engine  speed. 
Therefore  the  desired  engine  power  and  engine  speed  are  generated. 


Vehicle  Speed 


Figure  5.  Computational  flow  in  IEC-HEV  during  the  power  split  mode  at  every  time  instance  t. 
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During  online  driving,  at  the  beginning  of  a  drive  cycle  ( t  <A  Wrt),  IEC-HEV  uses  a  power  split  strategy  provided  by  the  default 
Ford  Escape  power  control  in  PSAT.  When  t>A  Wrt ,  at  every  time  instance  t=  A  WK1  +  j*At,  j  -  0,  . . until  the  end  of  the  drive 
cycle,  where  At  is  very  small,  e.g.  1  second,  IEC-HEV  obtains  the  current  vehicle  state,  represented  as  V_state  ( t )  =  [  v/t), 
Pdrive_sh(t ),  SOC(t) }  where  vs  ( t )  is  the  vehicle  speed,  P drive_sh(t)  is  the  driver’s  power  demand,  SOC(t)  is  the  state  of  charge  of  the 
battery,  uses  the  data  combined  with  the  vehicle  speed  data  from  last  a  few  seconds  to  generate  the  desired  engine  power  and  engine 
speed.  The  IEC-HEV  has  the  following  major  computational  steps.  First  it  calls  the  neural  network  NN_RT&TC  to  predict  the 
current  roadway  type  and  traffic  congestion  level,  and  the  neural  network  NN_DT  to  predict  the  driving  trend. 

NN_RT&TC  makes  the  prediction  based  on  the  fourteen  features  extracted  from  the  vehicle  speed  profile  during  time  interval 
[t-A  Wrj,  t],  where  A  Wrt  is  the  window  size,  which  was  discussed  extensively  in  [22],  They  are  distance  traveled  within  this  time 
interval,  the  maximum  speed,  acceleration  and  deceleration,  average  speed,  acceleration  and  deceleration,  standard  deviation  of 
acceleration,  %of  time  the  vehicle  traveled  within  speed  15  -  30km/h,  %  of  time  within  speed  >  1  lOkm/h,  %  of  time  in  deceleration 
within  interval  -10  ~  -2.5m/s2,  %  of  time  in  deceleration  within  intervals  -10  ~  -2.5m/s2,  -2.5  ~  -1.5m/s2,  and  -1.5  ~  -lm/s2,  and  the 
number  of  acceleration  shifts  within  this  time  interval.  The  output  of  NN_RT&TC  is  R,,  the  roadway  type  and  traffic  congestion 
level  predicted  at  time  t,  one  of  the  11  standard  Sierra  FS  drive  cycles,  1=1,.  ..,11. 

The  neural  network,  NN_DT,  was  developed  and  trained  to  predict  the  current  driving  trend  represented  in  one  of  five  states, 
no-speed,  low  speed  cruise,  high  speed  cruise,  acceleration,  and  deceleration.  The  driving  trend  reflects  the  driver’s  reaction  to  the 
traffic  condition,  therefore  is  an  important  factor  that  affects  fuel  consumption.  The  input  to  NN_DT  are  six  features  calculated 
from  the  past  vehicle  speed  profile  within  the  time  interval  \t  —  A WDT ,  t ) .  They  are,  the  average  speed,  maximum  speed,  minimum 
speed  and  average  acceleration,  during  the  time  period  [?  —  A WDT ,  t ) ,  the  vehicle  speed  at  ( t  —  A WDT ) ,  and  the  vehicle  speed  at  t. 

Since  driving  trend  can  change  in  very  short  time  interval,  a  small  window  size  should  be  used  here.  In  our  system  we  used  A  W* 
=  9  seconds.  The  design  and  training  details  of  these  neural  networks  were  presented  in  [22].  Based  on  the  roadway  type  and  traffic 
congestion  level  R  predicted  by  the  neural  network  NN_RT&TC,  IEC_HEV  calls  the  two  neural  networks,  AW'  and  aw'  , 

which  were  trained  for  generating  optimal  battery  power,  Pbatl(t ),  and  engine  speed  coeng(t )  respectively  for  the  roadway  type  and 
traffic  congestion  level  R,.  The  input  variables  to  NNL  are  { vs(f),  P drive  sh(t ),  SOC(t),  DT(t)  },  where  vs(t)  is  the  vehicle  speed, 

*bat 

Pdnve_sh(t ) 's  the  driver’s  power  demand,  DT(t )  is  the  driving  trend,  SOC(t)  is  the  state  of  charge  of  the  battery.  The  input  variables 
to  aw'  ,  are  {  vs(t),  Pdme_sh(f),  SOC(t)}.  Based  on  the  desired  battery  power  Pbanit),  and  engine  coeng(t),  we  calculate  the  following 

vehicle  parameters  can  be  calculated,  motor  power  P„w,(t ),  motor  power  loss  generator  power  Pge„(t),  and  generator 
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power  loss,  Pgenjoss(t),  as  explained  in  equations(6)-(10)  in  [22],  Then  the  desired  engine  power  Peng(t)  can  be  obtained  by  using  the 
following  equation: 


Peng(t) 

^ drive  ^  At)  +  PhJt)  +  Pimt_  loss^t)  Pgen_loss^^  Pl  ' 


(6) 


IV.  System  Implementation  and  evaluations 

The  intelligent  energy  controller,  IEC-HEV  has  been  fully  implemented  on  the  Ford  Escape  model  provided  in  the  PSAT 
library.  The  major  components  and  parameters  of  the  power  split  system  of  the  Ford  Escape  model  are  listed  in  Table  I.  Figure 
6  shows  integrated  IEC_HEV  in  the  vehicle  propelling  system  of  the  Ford  Escape  model.  IEC_HEV  is  placed  between  the 
Driver  Model  and  the  subsystem  control.  The  desired  battery  power  and  engine  speed  generated  by  the  neural  networks  are 
used  to  generate  torques  and  speeds  in  a  engine,  a  generator  and  a  motor  following  the  equations  given  in  the  Part  I  of  this  series 
[22], 


TABLE  I 

MAJOR  COMPONENTS  OF  THE  POWER  SPLIT  SYSTEM  IN  THE  FORD  ESCAPE  MODEL  PROVIDED  BY  PSAT 


Components 

Description 

Engine 

■  Ford  Escape  Hybrid  MY05  gasoline  engine 

■  2.3L  99kW 

Planetary  Gear  Set 
and  Driveline 

■  H-ing  =  29  Nsun  =  33,  Wheel  Radius  =  0.348m, 

■  Final  Drive  Ratio  =  3.77 

Generator 

■  Ford  Escape  Hybrid  MY05  permanent  magnet  motor 

■  Continuous  Power  =  17kW,  Peak  Power  =  33kW 

Motor 

■  Ford  Escape  Hybrid  MY05  permanent  magnet  motor 

■  Continuous  Power  =  33kW 

■  Peak  Power  =  65kW 

Battery 

■  Toyota  Prius  MY04  Battery  (ess_nimh_6_168_panasonic_MY04_Prius) 

■  Capacity  =  6. 5 Ah,  Cell  number  =168 

Vehicle  System  Control 


Subsystem  Control 


Ignition  Key 
Position 


Vehicle  Status 


Figure  6.  Implementation  of  IEC-HEV  in  the  Ford  Escape  model  provided  by  PSAT 
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In  order  to  conduct  an  in-depth  study  on  the  effectiveness  of  the  proposed  machine  learning  framework  ML_EMO_HEV,  we 
implemented  four  different  versions  of  the  machine  learning  algorithm  for  optimal  power  split,  IEC_HEV_SISE,  IEC_HEV_MISE, 
IEC_HEV_MIME  and  IEC_HEV_All.  The  neural  networks  were  trained  to  learn  optimal  solutions  generated  by  DP  based  on  these 
roadway  types  and  congestion  levels.  . 

•  IEC_HEV_SISE.  In  this  energy  controller,  for  every  Rt,  two  power  split  neural  networks,  NN'P  and  NN‘  are 

*bat  U'eng 

trained  with  the  data  along  the  single  optimal  path  generated  by  the  DP  optimization  algorithm  when  it  is  applied  to  the 
FS  drive  cycle  Rj,  with  the  initial  and  the  ending  battery  SOC  set  to  50%,  i=  1,...,1 1.  We  refer  this  training  method  to  as 
Single  Initial  point  and  Single  Ending  point  (SISE),  a  single  optimal  path  generated  by  DP  that  initiates  at  a  single  SOC 
point  and  ends  at  the  same  SOC  point.  Figure  7  (a)  shows  such  an  optimal  SOC  path  generated  the  DP  algorithm  for  the 
Sierra  FS  drive  cycle  Arterial  LOS  AB  cycle  (R8).  The  optimal  power  split,  optimal  battery  power  and  engine  speed, 
along  this  optimal  path  were  used  as  objective  values  for  training  the  two  neural  networks,  j\W'  and  NN‘  where 

Pbal  m eng 

i= 8. 

•  IEC_HEV_MISE.  In  this  energy  controller,  the  two  power  split  neural  networks  for  each  roadway  type  and  traffic 
congestion  level  R,  are  trained  with  the  data  from  the  multiple  optimal  paths  generated  by  the  DP  algorithm  applied  to 
the  Sierra  FS  cycle  represented  by  Rr  These  multiple  optimal  paths  are  generated  as  follows.  We  apply  the  DP 
optimization  algorithm  to  the  FS  drive  cycle,  Rj  multiple  times.  At  every  time,  the  DP  starts  at  a  different  initial  battery 
SOC  point  but  always  ends  at  50%  of  SOC.  The  optimal  power  split,  optimal  battery  power  and  engine  speed,  along  all 
these  optimal  paths  were  used  as  objective  values  for  training  the  two  power  split  neural  networks.  This  training  method 
is  referred  to  as  MISE  (Multiple  Initial  SOC  points  and  Single  Ending  SOC  point).  This  training  method  is  particularly 
feasible  for  real  time  vehicle  energy  management,  where  SOC  is  allowed  to  vary  in  the  range  of  [Si,  5?].  For  the 
experiments  presented  in  this  paper,  we  use  S /  =  0.4  and  S2  =  0.6.  For  this  reason,  we  used  the  DP  algorithm  to  generate 
multiple  optimal  paths  from  the  initial  SOC  points  at  0.42,  0.46,  0.48,  0.5,  0.52,  0.54,  0.56,  and  0.58.  All  paths  end  at 
50%  SOC.  Figure  7  (b)  shows  the  optimal  paths  generated  from  the  Arterial  LOS  AB  cycle  (R8).  The  same  procedure  is 
applied  to  every  Sierra  FS  drive  cycle  for  generating  optimal  power  split  points  used  to  train  the  power  split  neural 
networks  in  IEC_HEV_MISE. 

•  IEC_HEV_MIME.  In  this  energy  controller,  the  two  power  split  neural  networks  for  each  roadway  type  and  traffic 
congestion  level  R ,  are  trained  with  the  data  from  the  multiple  optimal  paths  generated  by  the  DP  algorithm  from  the  FS 
drive  cycle  Rj  with  multiple  initial  battery  SOC  points  and  multiple  ending  SOC  points.  This  training  method  is  referred 
to  as  MIME  (Multiple  Initial  SOC  points  and  Multiple  Ending  SOC  points).  Similar  to  IEC_EIEV_MISE,  we  assumed 
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an  SOC  operating  range  of  [5/,  S2]  =  [0.4,  0.6],  In  this  case,  the  DP  algorithm  generated  multiple  optimal  paths  from 
SOC  starting  points  at  0.42,  0.46,  0.48,  0.5,  0.52,  0.54,  0.56,  and  0.58.  Each  path  ends  with  at  the  same  SOC  points  as 
the  initial  SOC  point,  as  illustrated  in  Figure  7  (c). 

•  IEC_HEV_All.  In  order  to  study  the  effectiveness  of  roadway  type  and  traffic  congestion  level  specific  energy 
optimization,  we  trained  the  power  split  neural  networks  without  making  the  distinction  of  roadway  types  and  traffic 
congestion  levels.  The  DP  algorithm  was  applied  to  all  1 1  Sierra  SF  drive  cycles  with  the  initial  and  ending  SOC  set  to 
0.5.  The  resulting  optimal  power  split  points  were  used  to  train  two  power  split  neural  networks,  NN  „  and  WW 

‘ bat  a eng 

During  a  real-time  drive  cycle,  the  IEC_HEV_AIl  controller  applys  these  two  energy  control  neural  networks  to  all 
roadway  types  and  the  traffic  congestion  levels.  Figure  8  shows  the  configuration  of  IEC_HEV_All.  The  controller 
only  has  three  neural  networks,  the  neural  network  NN_DT  trained  for  predicting  driving  trend,  and  the  two  power  split 
neural  networks,  AW  and  AW  ■  WW„  and  NN  are  used  for  all  roadway  types  and  traffic  congestion  levels  to 
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generate  desired  battery  power  demand  and  desire  engine  speed  at  every  time  instance  t  during  real  time  vehicle 
operation. 

The  following  subsections  presents  the  experiments  used  to  evaluate  these  systems. 


Arterial  LOS  AB:  SISE  Optimal  Power  Split  Path  Arterial  LOS  AB:  MISE  Optimal  Power  Split  Paths  Arterial  LOS  AB:  MIME  Optimal  Power  Split  Paths 


Figure  7.  Optimal  power  split  path  generated  from  SF  drive  cycle  Arterial  LOS  AB  (a)  Optimal  energy  path  used  in  training  IEC_HEV_SISE,  (b) 
Optimal  energy  paths  used  in  training  IEC_HEV_MISE,  (c)  Optimal  energy  paths  used  in  training  IEC_HEV_MIME. 


13 


NN  ALL 


Figure  8.  Energy  control  in  IEC_HEV_All 


A.  Evaluation  of  IEC_HEV 

In  order  to  conduct  an  in-depth  study  of  the  power  split  control  generated  by  proposed  intelligent  energy  controller,  we  applied 
the  IEC_HEV_SISE  version  to  three  drive  cycles,  LA92,  NY_City,  and  UnifOl.  These  three  cycles  are  chosen  because  they 
represent  well  the  real  traffic  conditions.  Both  LA  92  and  UnifOl  consist  of  combinations  of  freeway,  arterial  and  local  roads  and 
various  levels  of  traffic  congestion  levels.  The  NY_City  is  a  typical  city  driving  cycle  with  many  brakes  and  a  number  of 
stop-and-go  scenarios. 

The  study  involves  the  analysis  of  the  performances  of  IEC_HEV  with  respect  to  the  accuracy  of  roadway  type  and  traffic 
congestion  level  prediction,  and  the  optimality  of  power  split  generated  by  IEC_HEV  and  represented  in  SOC,  engine  speed  and 
battery  power  during  the  drive  cycles,  and  fuel  consumption  over  each  drive  cycle.  The  performances  of  IEC_HEV  are  compared 
with  those  generated  by  the  Dynamic  Programming  (DP)  through  offline  computation,  which  is  often  used  in  literature  as  the 
benchmark  of  the  optimal  performance,  and  those  generated  by  the  Ford  Escape  controller  provided  by  PSAT  system. 

The  results  generated  by  the  three  systems  over  the  three  drive  cycles  are  illustrated  in  Figures  9-11.  Figures  9  -  1 1  (a)  illustrate 
the  roadway  type  and  traffic  congestion  level  prediction  made  by  IEC_HEV  over  the  three  drive  cycles.  Each  of  those  figures  show 
the  speed  profile  of  the  three  cycles  and  the  labeled  true  (in  red  color)  and  predicted  (in  blue  color)  roadway  types  and  traffic 
congestion  levels.  The  predictions  were  made  every  second  (Atrt=  1)  based  on  the  features  extracted  from  last  50  seconds  of  driving 
speed  (AWrt=  50). 

Figures  9  -  1 1(b)  show  the  SOC  changes  during  each  of  the  three  drive  cycles  generated  by  the  three  controllers,  DP  controller 


(in  red  color),  IEC_HEV_SISE  controller  (in  blue  color),  and  the  Ford  Escape  controller  in  PSAT  (in  green  color),  and  Figures  9  - 
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1 1  (c)  and  (d)  show  the  engine  speed  and  battery  power  curves,  respectively,  generated  by  the  three  controllers.  The  SOC  profile 
changes  provide  insight  to  how  the  controllers  manage  the  battery  power. 

For  the  drive  cycle  LA  92,  the  SOC  curve  generated  by  the  IECHEVSISE  was  closer  to  the  DP’s  SOC  curve  than  the  SOC 
curve  generated  by  Ford  Escape  Default  controller  from  t=  0  through  t=400  seconds.  During  t=400  seconds  and  t=480  seconds,  the 
IECHEVSISE  controller  was  charging  the  battery,  which  causes  the  IECHEVSISE’s  SOC  increased  2.54%  from  the  DP’s 
SOC.  Although  this  amount  of  deviation  was  carried  through  the  remaining  entire  drive,  the  IEC_HEV_SISE’s  SOC  still  closely 
followed  the  trend  of  DP’s  SOC  curve.  From  t=l 000  through  the  end,  the  SOC  curve  generated  by  the  Ford  Escape  Default 
controller  behaved  quite  irregularly  in  comparison  with  the  SOC  curve  generated  by  DP,  while  the  SOC  curve  generated  by  the 
IEC  HEV  SISE  still  closely  followed  the  trend  of  DP’s  SOC  curve  to  the  end.  The  ending  SOC  generated  by  IEC  HEV  SISE 
was  equal  to  the  starting  point,  which  is  50%,  while  the  ending  SOC  generated  by  the  Ford  Default  Controller  dropped  below  47 %. 
For  the  NY_City  drive  cycle  and  UnifOl  cycle,  the  IEC_HEV_SISE  followed  the  DP’s  SOC  curve  much  closer  than  the  default 
Ford  Escape  controller. 

Tables  II,  III,  and  IV  show  the  fuel  savings  by  Dynamic  Programming  and  the  IEC_HEV_SISE  controller  in  comparison  with 
the  default  Ford  Escape  controller  for  LA92,  NY_City,  and  UnifOl  cycles,  respectively.  All  experiments  started  with  battery  SOC 
set  to  50%.  However  only  the  DP  controller  ended  at  50%  of  SOC.  In  order  to  conduct  a  fair  comparison,  we  adjust  the  fuel 
consumption  based  on  the  ending  SOC  using  the  following  formula. 

Adjusted  fuel  Cost  —  [Yt=ifuel  rate(t)}  +  A(SOC(N)  —  SOCDP(Njj,  (7) 

where  t=N  is  the  end  of  a  cycle,  SOCdp  is  a  SOC  generated  by  DP  and  /.  is  a  fuel  cost  that  is  equivalent  to  1  %  of  SOC.  The  value 
of  X  is  determined  by  running  DP  on  the  same  drive  cycle  two  times,  each  ends  at  a  different  SOC.  For  the  drive  cycle  LA92, 
IEC_HEV_SISE  saved  over  9%  fuel,  for  NY_City  over  11%  and  UnifOl  over  8%. 

B.  Evaluation  of  four  different  version  of  IEC_HEV 

The  three  of  the  four  versions  of  IEC_HEV  controllers,  IEC_HEV_SISE ,  IEC_HEV_MISE,  IEC_HEV_MIME,  are  designed  to 
evaluate  the  effectiveness  of  the  optimal  paths  generated  by  the  DP  at  various  starting  and  ending  SOC  at  the  1 1  Sierra  FS  drive 
cycles.  The  fourth  controller,  IEC_HEV_All ,  is  designed  to  evaluate  the  effectiveness  of  roadway  type  and  traffic  congestion  level 
specific  power  split.  The  evaluation  of  these  four  controllers  are  conducted  based  on  their  performances  on  the  ten  test  drive  cycles 
provided  by  the  PSAT  library,  UDDS,  SC03,  HWFET,  Arb02,  NY_City,  Rep05,  LA92,  HL07,  US06,  and  UnifOl.  Figure  12 
illustrates  the  fuel  savings  of  the  four  controllers  along  with  DP,  the  optimal  controller,  on  all  ten  test  drive  cycles. 


Speed(rad/sec)  Speed  (rad/sec)  Battery  Power  (W)  Battery  Power  (W) 
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Simulation  plot 
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Figure  9.  (a)  Drive  cycle  LA92  labeled  with  true  and  predicted  roadway  types  and  traffic  congestion  levels;  (b)  SOCs  of  LA92  generated  by  the  three 
controllers;  (c)  Engine  speed  ;  (d)  Battery  power  comparison; 
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Figure  10.  (a)  Drive  cycle  NY_City  labeled  with  true  and  predicted  roadway  types  and  traffic  congestion  levels;  (b)SOCs  of  NY_City 
generated  by  the  three  controllers;  (c)  Engine  speed  ;  (d)  Battery  power  comparison. 
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Figure  11.  (a)  Drive  cycle  UnifOl  labeled  with  true  and  predicted  roadway  types  and  traffic  congestion  levels;  (b)SOC  comparison  with  UnifOl; 
(c)  Engine  speed  output  comparison;  (d)  Battery  power  comparison. 
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TABLE  II 

FUEL  SAVING  FOR  LA92 


LA92 

Fuel  consumption  before 
SOC  correction  (g) 

Ending 

SOC  (%) 

Fuel  consumption 
after  SOC  correction  (g) 

Saving  (%) 

PSAT 

898.90 

46.51 

916.36 

DP 

754.65 

50.00 

754.65 

17.65% 

IEC_HEV_SISE 

830.94 

50.00 

830.94 

9.32% 

TABLE  III 

FUEL  SAVING  FOR  NY_City 


NY_City 

Fuel  consumption  before 
SOC  correction  (g) 

Ending 
SOC  (%) 

Fuel  consumption  after 

SOC  correction  (g) 

Saving  (%) 

PSAT 

125.49 

47.57 

137.65 

DP 

108.84 

50.00 

108.84 

20.93% 

IEC_HEV_SISE 

117.42 

48.81 

123.35 

11.16% 

TABLE  IV 

FUEL  SAVING  FOR  UnifOl 


UnifOl 

Fuel  consumption  before 
SOC  correction  (g) 

Ending 

SOC(%) 

Fuel  consumption  after 

SOC  correction  (g) 

Saving  (%) 

PSAT 

1198.221 

47.23 

1212.085 

DP 

1013.148 

50.00 

1013.148 

16.41% 

IEC_HEV_SISE 

1075.642 

47.88 

1113.307 

8.15% 

Dynamic  Programming  cannot  be  used  for  in-vehicle  control  since  it  requires  a  priori  knowledge  of  the  entire  drive  cycle  and  is 
too  computationally  intensive  to  be  done  in  real-time.  Furthermore,  in  production  systems  such  as  the  Ford  Escape  or  Toyota  Prius, 
other  factors  such  as  drivability,  performance,  and  emissions  need  to  be  balanced  with  vehicle  energy  management.  For  this  reason, 
DP  only  serves  as  an  upper  bound  of  energy  optimization  in  a  vehicle  for  a  given  drive  cycle. 

It  appears  that  IEC_HEV_MISE  controller  has  the  best  performances.  It  outperformed  IEC_HEV_SISE  on  8  drive  cycles.  It 
outperformed  IEC_HEV_MIME  on  5  drive  cycles,  and  has  equal  or  close  performances  on  two  drive  cycles  (Rep05  and  NY_City). 
For  the  remaining  three  drive  cycles,  SC03,  HWFET  and  Arb02,  IEC_HEV_MISE  is  only  slightly  behind  IEC_HEV_MIME.  In 
comparison  with  IEC_HEV_All ,  IEC_HEV_MISE  has  large  leads  over  IEC_HEV_AII  on  all  drive  cycles  except  on  LA92,  on 
which,  IEC_HEV_MISE  has  similar  fuel  saving  as  IEC_HEV_All.  In  summary,  IEC_HEV_MISE  has  be  best  fuel  savings  over  all 
test  drive  cycles. 
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Figure  13  shows  the  ending  SOC  for  each  of  the  simulations.  The  ending  SOCs  of  IEC_HEV_SISE  in  all  ten  test  drive  cycles 
stayed  most  close  to  the  beginning  SOC,  which  is  50%  in  comparison  with  the  other  three  controllers.  IEC_HEV_MISE  comes  in 
the  second  place  in  terms  of  closeness  to  the  beginning  SOC  over  all  test  drive  cycles.  The  IEC_HEV_MIME  controller  generated 
large  swings  in  ending  SOCs.  In  most  test  drive  cycles,  IEC_HEV_All  ended  at  low  SOCs. 

If  fuel  savings  and  battery  management  are  considered  together,  we  can  conclude  that  IEC_HEV_MISE  is  the  best  controller,  and 
IEC_HEV_SISE  is  the  second  best.  Because  of  the  inferior  performances  of  IEC_HEV_AII  in  terms  of  both  fuel  saving  and  battery 
management,  we  can  further  conclude  that  machine  learning  of  optimal  power  split  based  on  roadway  types  and  traffic  congestion 
levels  is  an  effective  approach  of  intelligent  energy  management  in  vehicle. 


■  DP 

■  I  EC _ H  E V_SISE 

■  IEC_HEV_MISE 

■  IEC_HEV_MIME 

■  IEC  HEV  All 


Figure  12.  Comparisons  of  fuel  savings  by  Dynamic  programming  and  IEC-HEV  controllers. 
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Figure  13.  Comparisons  of  ending  SOCs  over  all  test  drive  cycles. 
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IV.  Conclusion 

The  choice  of  the  power  split  during  the  "engine  on"  mode  is  the  most  important  opportunity  in  the  vehicle  power  controller  for 
optimizing  fuel  economy.  Our  major  contribution  is  to  provide  intelligent  online  energy  control  while  the  vehicle  is  operating  in  the 
power  split  mode.  We  developed  and  implemented  three  intelligent  energy  controllers  under  the  proposed  machine  learning 
framework,  ML_EMO_HEV  ,  which  combines  Dynamic  Programming  with  machine  learning  to  learn  about  roadway  type  and 
traffic  congestion  level  specific  optimal  power  split  during  the  “engine  on”  mode.  The  details  of  ML_EMO_HEV  can  be  found  in 
the  Part  I  of  the  series.  All  three  online  controllers,  IEC_EIEV_SISE ,  IEC _ELEV _MISE ,  and  IEC_HEV_MIME,  consist  of  three 
major  types  of  neural  networks,  a  neural  network  trained  to  predict  roadway  types  and  traffic  congestion  levels,  a  driving  trend 
prediction  neural  network,  and  22  power  split  neural  networks  trained  specifically  for  the  1 1  roadway  types  and  traffic  congestion 
levels  defined  by  Sierra  Research.  The  output  from  the  power  split  neural  networks  are  optimal  engine  speed,  coeng  (t),  and  optimal 
battery  power,  Pb„n(t),  which  are  then  used  to  generate  the  optimal  engine  power,  Peng  (t).  In  turn,  both  Peng  ( t )  and  coeng(t)  are 
checked  to  make  sure  they  are  within  the  appropriate  constraints. 

The  three  IEC_HEV  controllers  have  been  implemented  inside  the  Ford  Escape  model  in  PS  AT  for  performance  evaluations . 
Their  performances  on  the  10  test  drive  cycles  provided  by  PSAT  are  analyzed  and  compared  with  the  DP  offline  optimal 
controller  and  the  default  controller  used  in  the  Ford  Escape  model  provided  by  PSAT.  We  also  evaluated  the  effectiveness  of 
roadway  type  and  traffic  congestion  level  specific  power  split  by  comparing  the  performances  of  the  three  controllers  with 
IEC_HEV_All ,  an  IEC_HEV  controller  containing  two  power  split  neural  networks  applicable  to  all  roadway  types  and  traffic 
congestion  levels.  Based  on  these  experiments,  we  can  make  the  following  conclusions. 

•  The  IEC_HEV  controllers  trained  with  the  proposed  machine  learning  framework,  including  the  IEC_HEV_All, 
provide  substantial  savings  on  all  drive  cycles.  The  fuel  savings  range  from  3%~20%  over  the  default  Ford  Escape 
model  for  the  ten  PSAT  cycles. 

•  The  roadway  type  and  traffic  congestion  level  specific  trained  IECJHEV  controllers,  IEC_HEV_SISE, 
IEC_HEV_MISE,  and  IEC_PIEV_MIME,  in  general  have  better  performance  than  the  general  IEC  controller, 
IECJiEV_All. 

•  The  best  controller  is  IEC_HEV_MISE.  It  is  a  roadway  type  and  traffic  congestion  level  specific  intelligent  energy 
controller  trained  with  multiple  optimal  paths  generated  by  DP  with  multiple  initial  SOC  points  and  the  one  ending 
SOC  at  50%.  This  controller  provides  substantial  fuel  savings  and  stable  SOC  control  in  all  drive  cycles. 
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